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We discuss an analysis of Constraint Satisfaction problems, such as Sphere Packing, K-SAT and 
Graph Coloring, in terms of an effective energy landscape. Several intriguing geometrical properties 
of the solution space become in this light familiar in terms of the well-studied ones of rugged (glassy) 
energy landscapes. A 'benchmark' algorithm naturally suggested by this construction finds solutions 
in polynomial time up to a point beyond the 'clustering' and in some cases even the 'thermodynamic' 
transitions. This point has a simple geometric meaning and can be in principle determined with 
standard Statistical Mechanical methods, thus pushing the analytic bound up to which problems 
are guaranteed to be easy. We illustrate this for the graph three and four-coloring problem. For 
Packing problems the present discussion allows to better characterize the 'J-point', proposed as a 
systematic definition of Random Close Packing [l|], and to place it in the context of other theories 
of glasses. 



PACS numbers: PACS Numbers : 75.10.Nr, 02.50.-r,64.70.Pf, 81.05.Rm 

I. INTRODUCTION 

Constraint Optimization and Satisfaction problems are a particular yet widespread class. The prototype is the 
Packing Problem, in which we are given a fixed volume V and asked to pack A'^ objects of typical size Tq (a scale 
factor) without overlap, making N or Vq as large as possible. When Tq is increased (with N, V fixed) the problem 
becomes harder as the number of configurations that solve it decreases, until a point is reached rg"'^*^ beyond which 
there are no more solutions. 

Another celebrated example is the satisfiability, or SAT, problem Q: we have a set of N Boolean variables {xi ~ 
0, l}i=i,...,Ar and a number M = aN of clauses — when all clauses have exactly K literals, the problem is refereed to 
as K-SAT — which in the present paper we shall assume are the first M of a longer list that is generated at random 
and stipulated once and for all. We are asked to find logical assignments satisfying 

M M 

F=A^^=A (4fvz(fv...Vzlf) , (1.1) 

where /\ and V stand for the logical AND and OR operations, respectively, £ labels a set ii,...,iK and zl^'' is, 
depending on £ either Xi or its negation (this, and the Xi that participate in each factor is what defines the clause). 
An assignment of the {a;,;}'s satisfying all clauses is a solution of the K-SAT problem. When the number of clauses 
aN = M increased, the number of sets {x^j's satisfying all clauses decreases, until a point q-""'*''* is reached beyond 
which there are no solutions. 

The third problem we shall consider is the Graph g-Coloring Problem Q. We are given a graph with N vertices 
and are asked to color them with one of q colors so that no two vertices sharing a link have the same color. We shall 
assume as before that we have a list of links, and we count the number of possible colorings when the graph has the 
first M = 2q!A^ of the list (where the average connectivity of the graph is c = 2a). As M is made larger, a point 
cluneal is reached in which no more colorings are possible. 

Coloring problems are a particular class of packing problems. Consider the following 'angle-packing' construction: 
in every node i of a graph we assign an angular variable 0i. There is a repulsive potential V{9i — 9j) between the 
angles of linked sites as in Figure (TJ it is nonzero where the angle difference {8i — dj) lies between —2TT/q and +2Tr/q 
(modulo 27r). For every angle configuration with zero total energy, one can obtain a solution of the q-Coloring problem 

just by assigning to the site i the color numbered by the integer R = Int (^^^ {R = 0, 1, q — 1). Conversely, each 

coloring solution has at least one zero energy configuration counterpart. 

Statistical Physics methods arc particularly efhcient in treating large random problems, especially when mean- field 
techniques are exact, and indeed several important results where obtained for both satisfiability 0,13 ^-nd coloring^ 
in this case. The connection with statistical mechanics is usually introduced as follows (see for example [1, 0,W)' 
one first defines an energy E such that it is zero when the problem is satisfied and larger otherwise. In the examples 
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FIG. 1: Potential of soft particles with finite-range interactions. It allows to construct a packing version of the coloring problem 
(see text). 

above, this could be the number of overlapping spheres, the number of unsatisfied clauses, and the number of links 
joining vertices with the same color, respectively. Next, one studies for finite temperature T = 1//3 the partition sum 
over configurations 

Z=Y, e-/^^^^""-^) (1.2) 

conf 

The results are encoded in the energy and entropy in the zero-temperature limit. One can apply all the analytical 
techniques of Statistical Mechanics, and has in addition available all the standard practical methods of annealing 
energies (in metallurgy as well as in computational physics) that serve at least as a first, general purpose numerical 
tool. 

In this paper we shall follow a different strategy. We first introduce in section |TT] a pseudo-energy defined directly in 
terms of the Constraint Satisfaction problem. For the packing problem, consider the set of configurations that satisfies 
the constraints at a given value of Tq. If we make the problem harder by increasing the common scale of all particles 
To, the set of configurations still satisfying the constraints is strictly a subset of the previous one. Thus, we can 
construct a single- valued landscape function £ as in Fig [S] For the SAT and Coloring problems we proceed similarly, 
with now a playing the role of Tq, increased by adding clauses {resp. links) one by one following the predefined list. 
The variable conjugate to Z is in fact a pressure. 

Next, we explore in section IIIII the consequences of making the following Landscape Hypothesis: The pseudo- 
energy £ landscape defined above has in the thermodynamic limit the same qualitative properties as the usual rugged 
energy landscapes. This means that pressure plays a similar role than that of an inverse temperature. We shall find 
that several intriguing results in the geometry of random Constraint Satisfaction Problems (CSP) such as the 
satisfiability of random formulae or the coloring of random graphs can be readily understood in this language. 

In section IIV[ we describe a family of algorithms that immediately suggests itself: it is simply a quench in the 
pseudo-energy landscape, and hence by construction polynomial in the system size. At each step one increases the 
difficulty of the problem, and moves to a configuration in a finite neighborhood so as to restore satisfaction, if there is 
no such configuration the program stops. We shall show that its performance — where it converges and in what times 
— can be in principle determined analytically for the very same problems for which analytical statistical mechanical 
solutions are available. The method can be generalized to slower annealings, or to finite pressure. 

Next, we explore in detail the performance of this algorithm for the random q-coloring problem. Although here we 
are more interested in constructing a benchmark than a competitive numerical strategy, we find that the performance 
is surprisingly quite good. 

In section fVl we concentrate on hard spheres and recognize that this algorithm yields the procedure of O'Hern 
et al. [l| (and more generally the Lubachevsky-Stillinger jl^ procedure if a slower annealing is made) to define the 
so-called 'J-point', which they propose to identify as Random Close Packing. Comparing with the random satisfaction 
problems of the previous sections, we are thus able to give a mean-field realization of the J-point scenario [ssj. and to 
put it in the context of other special points in glass theory, such as the (putative) ideal thermodynamic glass state, 
and the (finite-dimensional relic of) the Mode-Coupling transition. 
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FIG. 2: Constructing a pseudo-energy from a constraint satisfaction problem: the regions shown are the satisfied sets at each 
level of difficulty. The height function is such that at each level the intersection corresponds to a constraint satisfaction problem. 

II. CONSTRAINT SATISFACTION PROBLEMS AND A LANDSCAPE HYPOTHESIS 

Let us describe the pseudo-energy in more detail. Consider a fixed set of particles of any shape whose positions 
and angles are given by the coordinates xi, ...,xjv, with a common scale ro, such that for example multiplying Tq by 
two doubles the linear size of all particles while preserving their shape. According to Fig. 1, the landscape at a point 
-E(xi, ...,XAr) is defined as (minus) the first value of Tq at which there is a particle overlap for these coordinates. (The 
minus sign is conventional). 

In particular, for spheres of equal size Tq, the pseudo-energy is simply (minus) the smallest inter-particle distance 
in that configuration divided by two. As such, it is very close to the succession of potentials discussed by Stillinger 
and Weber 



Esw 
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for ?i — > (X) 



(II.3) 



Here we are considering 



Xa - Xfcl 



(11.4) 



(The factor is needed to assure that the quantity is extensive if we make the limit n ^ oo before the thermodynamic 
limit.) Note that a gradient descent in 3\) follows the same path as one in ( f//.^[ ). 

For the SAT and the Coloring problem the space is discrete, and the difficulty of the problem may be increased by 
adding clauses and links, respectively. The pseudo-energies are then, by analogy, minus the number of clauses, and 
minus the numbers of links, respectively. As mentioned above, we are considering in fact a succession of problems, 
obtained by choosing the first M of a list of clauses or links which is fixed once and for all. Note that if we could 
reshuffle the list, the net result at finite M would depend on the length of the list itself: for an infinite list we would 
optimize the choice of M constraints to make them easily satisfiable — this is called the 'annealed' problem in the 
disordered system literature (not to be confused with the 'annealed approximation' of the probabilistic literature, 
which is something else altogether [l^ ). 

The landscape for the Coloring problem is very rugged indeed. We have a graph with M links that is well colored. 
Let us consider how the pseudo-energy changes when we flip the color of one given vertex. The new pseudo-energy is 
(minus) the largest value of M' that will make the new graph colorable, so we have to take away one by one the links 
in reverse the order, starting from the last introduced from M to M' , but most of the links deleted will not be even 
close to the configuration we have fiipped, and it will typically take M — M' ^ 0{N). We shall see that this poses 
no problem if one proposes changes only in nearby sites along the tree: thus, jumps will be always to configurations 
that are close and have lower pseudocnergy, avoiding other ones in the neighborhood that arc much higher. 
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From constraint satisfaction to energy landscapes and back 



As mentioned above, in this paper we shall make the Landscape Hypothesis that the pseudo-energy defined above 
has in the thermodynamic limit the same qualitative features as expected from a generic rugged energy landscape. 
We do not know if this is strictly so, but it is clear that the converse is true: every rugged energy landscape yields, in 
the microcanonical ensemble, a constraint satisfaction problem (isj . This is easy to see just reading Figure [2] in the 
reverse way: we imagine that the landscape i?(x) is given, and by cutting (microcanonical) slices of fixed energy at 
different depths £, we obtain the CSP of finding x such that H{x.) < £. 

What we shall do in the following section is to briefly review some known facts from complex energy landscapes, 
and then proceed by cutting microcanonical slices to infer the behavior it implies for random Constraint Satisfaction 
Problems. 



Let us review what we have learned, mostly from mean-field analytic calculations, about rugged energy landscapes. 
In order to match the constraint satisfaction situation, wc shall adopt the rather unusual point of view of making energy 
slices, rather than constant temperature ones. The states we encounter shall then be regions of the energy shell that 
are dynamically isolated. Figures |4l [5] and [6] show various of these constant energy slices: the temperature T is then 
a variable that classifies the states at that energy: in the landscape of £ it amounts to classifying states according to 
their pressure (the inverse of the temperature conjugate to £). This is not the usual practice in optimization problems, 
which is to classify the states according to their internal entropy a (the logarithm of the number of configurations, 

1 _ 9a- \ 



In what follows, we shall frequently allude to 'states', defined as regions of phase space that arc dynamically isolated, 
in the sense that the system starting a random walk within them will take a long time to exit. 'Long' might mean 
'infinite', or simply diverging with the system size N faster than any power law. Consider for example a ferromagnet 
with continuous spins on a lattice of dimension d > I and linear size L, with nearest-neighbor interaction 



In terms of the temperature, the phase diagram looks as Figure [31 there is a critical temperature below which there 
are two states of magnetization A/ = ^ si ~ ±mU^ , and above one state of zero magnetization. As is well known, a 



III. ENERGY LANDSCAPE: MICROCANONICAL SLICES 



A. Energy vs. Free energy barriers 
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FIG. 3: Energy vs. Temperature of a ferromagnet 
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system at Ti < Tc performing Monte Carlo dynamics starting with magnetization density m will take a time to jump to 
the state of magnetization — m that diverges in the thermodynamic limit. We can also make a random walk at constant 
energy Ei, and the time to pass from the +m to the — m state is still divergent. Does this mean that the constant 
energy surface E{s) = Ei is disconnected in two regions? A moment's thought shows that this cannot be the case. 
The actual saddle point in energy separating the two energy minima is just the energy to create an interface between 
half lattice at magnetization +in and half at —m, Esaddie ~ Eo ^ L''^^, while Ei — Eo ^ 0{L^) >> Esaddie ~ Eo 
(see Fig. [3]). We conclude that the energy surface is actually guitar-shaped, and the reason why the system takes a 
divergent time to pass from one state to the other is that (for large size L) the neck is exponentially thin. 



Consider a system of particles with a finite-range potential as in Figure [U confined to a fixed volume. To the extent 
that the packing without overlaps is possible, it is clear that the total energy landscape will have minima of zero 
energy which are flat. Surprisingly (and confusingly) also the pseudo-energy landscape can have 'flat bottom' states, 
both in the particle and in the K-SAT and Coloring problems, as we shall see in the next section. They turn out to 
be very important. 



We shall now give three examples of mean-field landscapes with many states. Mean-field problems in physics 
correspond to large random problems in optimization and satisfaction: the coloring of large random graphs or the 
satisfiability of random formulas are instances of this class. Indeed these are the models for which we expect a 
correspondence between landscapes to hold. 

The first example (the Sherrington-Kirkpatrick model) has the property that its states having free-energy density 
higher than the ground state one are no obstacle for the dynamics. This class also includes all models with smooth 
short range interactions in finite dimensions, like the Edwards Anderson model. 

The second example, the p spin model is the precise opposite: a naive dynamics is trapped in a 'threshold' level 
where the first, highest minima are encountered — this happens at energy density above the lowest. 

The third example, the Isingp-spin model, is the most typical of random optimization problems: it has a combination 
of 'transparent' high states that are avoided by the dynamics, but it also has a threshold level at finite energy density 
above the lowest, below which the dynamics only goes in times that diverge with the size. 



The Sherrington-Kirkpatrick model [8| (SK) is the most studied spin-glass, it consists of ±1 spins coupled by 
random, fully connected interactions 



It was soon realized [lalial that the free energy landscape had many metastable states (see (3), a situation depicted 
in Fig. m The dynamics in the thermodynamic limit has been also solved and surprisingly one finds that at 
each constant temperature the energy density tends for large times to the equilibrium one. In other words: all but the 
very lowest metastable states are transparent to the dynamics, if one wishes to compute the energy per spin with any 
percentage accuracy, this can be done in polynomial time (although the true ground state might might be harder to 
find). It took over a decade to clarify completely this from a pure 'landscape' point of view [20[. 

Because the states are not strong dynamic traps, a perturbation with small random forces that do not derive from 
a potential - a weak 'stirring' — immediately sets the system into motion: the whole state structure is washed away. 



B. Flat bottom states 



C. Mean- field glass landscapes 



1. The Sherrington-Kirkpatrick class 




(III.6) 



2. Spherical and Ising p-spin glass. 



The spherical p-spin model (p > 2) has a Hamiltonian: 




(III.7) 
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FIG. 4: A sketch of the metastable-state space of the Sherrington-Kirkpatrick modeh the gray area in the graph corresponds 
to a region with an exponential number of metastable states. In the inset: complexity along a constant-energy surface line A 
in terms of temperature T (the same graph would be obtained by plotting in terms of state entropy a.) 



with the Si satisfying a spherical constraint. It has a well studied [23, [2J] structure of states shown in Fig. [H Each 
line in the figure symbolizes a state (a free energy minimum). These do not merge and keep their order. There is 
a threshold level above which all the stationary points are unstable. States under the threshold are stable, the more 
so the deeper, and those that are just below are marginal, the 'spin glass susceptibility' diverges within them [2^ . 
A system sitting on the threshold states will be set into motion by small non-conservative 'stirring' forces (another 
manifestation of their marginality) , but this is not the case with deeper states which are 'rigid'. 

The zero-temperature intercept of the threshold energy E* will be of particular importance for us. Just above this 



point are the overwhelming majority of energy barriers 25[, and this means, according to the discussion above, that 
for all energy slices higher than E* the states are in fact connected by narrow bridges. 



E* 
E„ 




FIG. 5: The structure of metastable states of the spherical p-spin model. 



The out of equilibrium dynamics of this model is well studied [26|, a quench starting from a random configuration 
never goes below the threshold level. At zero temperature, a quench ends in the energy E* . In fact, no (instance 
independent) external field protocol is known that leads below the threshold: temperature cycling, magnetic field 
changes, even quantum annealing [28j have been tried, but the system always ends up on the threshold level at the 
end of the cycle. 

The thick line of Fig. [5] labeled 'paramagnet' corresponds to the equilibrium high-temperature phase. Above Ed 
this is a big state that dominates the dynamics and contains the overwhelming majority of configurations. Between 
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Ek and Ed the 'paramagnet' is fractured in many states sharing each a negligible volume, the probability of two 
configurations chosen at random of being in the same state is zero. Below E^ the lowest states dominate, there are 
still many states having a non-negligible volume, but the sharing of volume is so inegalitarian that two configurations 
chosen at random have a finite probability of being in the same state. 




T 



FIG. 6: A sketch of the organization of states in the Ising p-spin model. In the inset: complexity in slice C; the two vertical 
lines indicate the dynamic threshold (left) and the typical configurations (right), the dashed line symbolizes states that are 
transparent to the dynamics. The situation in slices A-E have been all found in constraint satisfaction problems for different 
levels of constraints (see text). 

Microcanonical slices at different energy levels give us a first glimpse of what happens in a constraint satisfaction 
problem. The number of states, as calculated in j2^, grows with decreasing temperature (at constant energy), up 
to r = — or the intersection with the threshold line, if this happens first. A system prepared at any energy 
E* < E < Ed performing energy-conserving dynamics [27| will end in the threshold level. If, on the contrary, E > Ed 
the dynamics will evolve in the paramagnetic state. 

Note the presence at Ed < E < Etap, in a range in which the paramagnet dominates, of states that are completely 
irrelevant from the dynamic (or static) point of view: these where found in IboI . [sH ] for the coloring and SAT 
problems in a given range of parameters. 

The spherical p — spin model is somewhat special, and in fact the generic situation is richer. For example, the 
Ising version of the p-spin model has a structure sketched in figure [H] In addition to the states that resemble those of 
the spherical case, there is a set of high-lying states that resemble those of the SK model [s^ and are transparent to 
the dynamics. A small 'stirring' perturbation wipes out the high-lying states, while the deep non-marginal ones are 
stable. There is still a dynamically defined threshold level above which states are marginal [23, but it is still not clear 
if there is a purely 'local' characterization of it (but at any rate an exact dynamic calculation is always possible p^). 
We shall in the last section also propose a static - though non-local - calculation to achieve the same goal. 

Making microcanonical sections as in figure [SI we see some of the possibilities for the organization of states and their 
respective dynamical and equilibrium properties. The inset of Figure [Hi shows a sketch of section along the line C: 
the states with smallest internal entropy correspond to the 'gray' region of dynamically transparent 'SK-like' states. 
This feature has been found in K-SAT [3^ and in the Coloring problem [sj . The typical configurations (vertical line 
to the right in the inset of Fig[5|) are those that maximize a + T, and do not coincide in general with the threshold 
states [38| . 

In any case, the level achieved with a zero-temperature annealing E* may (and as we shall see will) be lower than 
Ed and even Ek- For example, in the p-spin model these points can be calculated analytically [H,[2^ and the results 
arc plotted in figure [7| 

The important conclusion of this figure is that if the task is to find a configuration with energy as small as possible, 
generically neither Ed nor Ek pose a serious limit in themselves , because E* may be smaller. The same is true, for 
the same reason, for constraint satisfaction problems. This is the more so the smaller the values of p - and eventually 
K and q in SAT and Coloring. We shall see this in the Coloring Problem in the next section. 
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FIG. 7: The energies Ed, Ek, E* and Eo for the p-spin model as a function of p (from [3J|). A simple gradient descent program 
goes to E* which is beyond Ed, Ek for p < 13. 

3. Random constraint satisfaction problems 

The random satisfiability and the random graph coloring problem have been studied using the cavity method 0, @| 
and their landscapes when varying the connectivity was characterized in [s^, [3l| . It is interesting to observe that 
they follow our landscape hypothesis: they behave indeed like microcanonical slices of the typical energy landscapes 
we have been discussing. This is best seen in figure |8l 
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FIG. 8: Sketch of the set of solutions in random CSP when the connectivity is increased (adapted from [3, [s^l). From left to 
right, in order of increasing difficulty 1) easy problem, the whole solution space is connected, 2) a negligible fraction of solutions 
become disconnected, 3) 'Dynamic transition' ad'- the whole space breaks into dynamically isolated regions, two configurations 
at random belong to different regions, 4) Replica Symmetry Breaking, Kauzmann point ax- the regions become so rare that 
the probability of two configurations being in the same region is now non-zero, 5) Equilibrium 'rigidity point' q. Typical 
configurations have frozen variables. 6) Best packing, unSat, unCol {cy.pack, ctunsoi, ctuncoi)- the last configuration satisfying the 
constraints disappears. Red and grey regions symbolizes clusters with and without frozen variables, respectively (color online). 

For the coloring problem, the col-uncol transition appears at a„nco;(3) = 2.355 and q;mtico/(4) = 4.45 (Hj. The other 
critical connectivities in the properties of the landscape were found and computed recently by [soj . Using 3 colors, 
one has ad(3) = 2 (a result already obtained in [12), Qfif (3) = 2 and q;'"""'*(3) = 2.33. For the 4-coloring problem, 
the critical values are ad(4) = 4.175, a/f (4) = 4.23 and a'"'''''(4) = 4.42. 



IV. ALGORITHMS: ARKLESS STRATEGY FOR FLOOD VICTIMS 



Consider a rugged landscape that is being slowly flooded. We adopt the following strategy: we stay still until the 
shore reaches our feet. We then move in small steps away from the water, just enough each time to stay dry. At 
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a certain point, the patch where we are standing becomes an island, all land bridges have been flooded: this is the 
'dynamic transition level'. We keep on moving uphill, just avoiding the waterline. Our island may further divide into 
smaller ones, but our choice of island is dictated by the evolution of the shoreline. Our fate is sealed when our island 
disappears altogether: this may happen all of a sudden if its top is flat, or gradually if the top is rounded. Note that 
nothing guarantees that we have ended in the highest summit, so our survival level is just locally — but not globally 
— optimal. Clearly, the 'dry land' is the set of configurations satisfying the constraints, and the 'level of the water' 
the number of clauses, links or particle size, which wc increase gradually, while at the same time moving in small 
steps to remain satisfied. The point reached with this algorithm is image (5) of Fig. (S) 

This is the 'fast' procedure, a slightly more sophisticated one is at each level of the water, not just to stay at the 
shoreline, but to explore randomly all the available land, always without crossing water. This means that on occasion 
we shall take advantage of a land bridge to go (randomly) from a promontory to another. 

A composite strategy is to explore all the available land at one level of water, and then for all subsequent levels 
just follow the direct 'lazy' strategy of moving just what is necessary. 

Now, from the point of view of pseudo-energy landscape (which is reversed with respect to the flood analogy, with 
summits becoming valleys), in continuous cases it is clear that the 'fast' algorithm is just a gradient descent, and in 
general a zero temperature quench, while the procedure in which wc take time to diffuse at constant energy, is a 'slow 
annealing' [5^ in this case we have to specify how slow. The last, composite strategy, is again a rapid quench, but 
this time starting from an equilibrium configuration at some level. 

A. Hard sphere procedures. 

For the packing of hard spheres, the two procedures mentioned above have been used by Lubachevsky and Still- 
inger [lo| (the slow annealing), and by O'Hern et al. [l| (the rapid quenched). In both cases one 'inflates' all spheres 
simultaneously by a very small amount, and then uses some repulsion to eliminate any overlap (wet toes, in the 
analogy above) that might have been generated 0- In the annealed version, one also takes time to diffuse without 
changing the radii or allowing overlaps. 

Another possibility is to adapt the composite procedure used by Sastry et al. [s^ to analyze (true) energy landscapes: 
one starts from a fully equilibrated configuration at a given (low) level of packing, and from there onwards performs 
a fast quench, without diffusing at each step more than is necessary to eliminate overlaps. 

The reason why O'Hern et al. used the 'rapid quench' rather than an annealing is that their purpose was to 
define a point in parameter space (the 'J-point') in an unambiguous way. Had they used an annealing procedure, a 
different point would have been obtained for each annealing time — not to speak about problems of crystallization in a 
mono-disperse sphere case (and, indeed, the infinitely slow annealing limit would lead to a packing fraction essentially 
corresponding to equilibrium). 

As mentioned above, one of the aims of this paper is to put this 'J-point' in the context of the rest of glass theory. 
Wc shall discuss this in further detail in section |Vl Let us note here that we can apply exactly the procedure of 
O'Hern et al. (or Lubachcvsky-Stillinger) for the 'angle-packing' problem described in the introduction for a given, 
fixed graph, starting from q — oo and decreasing q. The smallest integer value of q attained gives then a realization of 
Coloring Problem. We shall not pursue this line here, but rather attack the coloring problem from a different angle. 

B. Introducing temperature in pseudoenergy landscape. 

The pseudoenergy landscape suggests how to construct a Monte Carlo/Parallel Tempering program for hard spheres. 

Consider monodispcrse hard spheres - the generalization to polydispcrsc is straightforward ~ at 'inverse temperature' 
(i.e. pressure) (3. Given a configuration, its pseudoenergy is proportional to (minus) the smallest interparticlc distance 
2ro. We choose a particle at random and displace it by a random amount. The minimal interparticlc distance 2r°''' 
may have changed: if it is so it must be due to the distance between the particle just moved and one of its neighbors 
becoming some value r"*^™ < r"'"'^ We thus accept the motion if r"^"' > r°''^, otherwise we accept it with probability 
g-2/9(r^ -'"o y This is the Monte-Carlo procedure. One can now run several copies in parallel at different /3 and 
implement the usual Parallel Tempering procedure. 

C. Discrete lattice procedures 

We have chosen to exemplify the algorithm in the random g-coloring problem. Wc arc given a long list of links, and 
add them one by one. Suppose that wc have a graph with M — 1 links, and a configuration of colors for the vertices 
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FIG. 9: Adding a link to a tree. 

satisfying the constraints. We now add the M — th link from the list: with probability one it will locally look as in 
Fig. [5] As we add the link, it may be that vertices A and B are of different colors, in which case we proceed. If, on the 
contrary, the colors of A and B coincide, we have to modify at least one of the two. This will typically create problems 
in the tree that has roots in them, so we shall need to modify colors up the branches. If this can be done with a finite 
number of trials, we do it, compute the total number of steps it took, and proceed with the next link addition. In 
practice, for the readjustment of the trees with root A or B , we use a Walk-COL algorithm introduced in [s^l, which is 
the coloring version of the celebrated I^a/fc-S'.AT program [U, [iO] This is perhaps not ideal, but it definitely provides 
an upper bound of the number of flips needed. Moreover, this strategy is straightforward to implement numerically. 
An important property of this algorithm is that it is focused: it changes only frustrated variables and it thus acts 
locally starting from the initial frustrated link, and does not perturb the solution far from this link. 

In this form, the algorithm is a recursive version of the method called Incremental- SAT [57| in the computer science 
community, in which one starts from a SAT formula and its known solution (or a graph and its proper q-coloring) 
and then adds one formula (or a new link for coloring) and has to find a solution to the new problem. This is usually 
done using Walk-SAT, as we do, although this procedure has not been proposed as far as we know as a general way 
to obtain solutions starting from scratch. The point reached by our algorithm can thus be seen as a EASY/HARD 
transition for the recursive, incremental COL problem. 

D. Coloring random graphs 

We now discuss our results and show our data for the 3-coloring and the 4-coloring in fig [TUl It turns out that the 
number of steps needed (the number of re-colorings close to A or B to get back to satisfaction) grows on average and 
diverges at a given connectivity a*, where our program stops. 

The fact that the rearrangement needed grows with the connectivity is not surprising in view of the results of Mon- 
tanari and Semcrjian [43l. |44|. who showed that the minimal number of rearrangements needed to satisfy constraints 
following a change of color of a random vertex diverges like a power law at a certain connectivity. The distance along 
the graph of the rearrangements is a power law as well. This critical connectivity corresponds in fact to the rigidity 
transition q-'"""'^ considered in [s^l where frozen variables (or 'hard fields' in the cavity language) appear that fix the 
value of a finite fraction of variables for all configurations within a state. After a situation with frozen variables is 
reached, an additional link will have a finite probability of connecting two variables frozen to the same color, thus 
rendering unsatisfied all configurations within the state. States with hard fields have a high mortality rate: after an 
extensive number of link additions, the survival probability of such a state is exponentially small in N . 

In fact, here the situation is somewhat different to the one in [H, [3], in that we are not considering an equilibrium 
configuration, but one obtained by a succession of link additions and rearrangements. The actual value a* that our 
program can reach need not coincide exactly with a'"""'^, because hard fields appear at somewhat lower connectivities 
in our out of equilibrium procedure. 

It can be shown that for large connectivity, all clusters have frozen variables beyond a certain connectivity (see for 
instance [s^, [3l| or [g^I for rigorous proof in SAT): this puts a strict limit to the algorithm. For the 3 and 4 coloring 
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FIG. 10: Performance of the recursive algorithm for the q = "i, 4-random coloring problem. The time needed to find a proper 
coloring diverge at a connectivity q, < auncoi in both case, but the algorithm is able to find colorings in linear time beyond 
the clustering ad and Kauzmann qa' transitions. 



problem, it has been shown (see again [30|) that all states have frozen variables for a > a'""''^. 

Because we can scale the curves with system size, and we consider only the regions where the curves superpose 
properly, we can in fact infer the behavior for N ^ oo. We do that in fig. [TlJ In the 3— coloring, the algorithm 
reaches in time linear in N a value a* w 2.275 which is beyond the clustering and Kauzmann transition ad = olk ~ 2. 
It is however, as expected, below the appearance of frozen variables in the thermodynamic states that take place at 
the rigidity transition — 2.33 (and of course below a^ncoi = 2.345). 

In the 4-coloring, the algorithm reaches in time linear in A'^ a value a* w 4.31 which is again well above Ud ~ 4.175 
and aK = 4.23 (but still systematically below q-'''""'^ = 4.42 and auncoi = 4.45). 

The integrated number of steps up to connectivity a is (a* — a)^'' with /i ^ 0.28. In Figure [TT] we show for reference 
the case q = 3, which is somewhat more dubious because it is not clear yet to which class - from the replica theory 
point of view - it belongs. In any case, the value a* 2.275, far beyond ad — ax = 2, is quite close (but still smaller) 
to the one {asp ~ 2.3) obtained in Ref Q with the best algorithm — Survey Propagation (SP) (3] — for smaller sizes. 




t t 



FIG. 11: Estimate for the asymptotic a* of the algorithm. Left: q — 3-random coloring problem. Right: q = 4-random coloring 
problem. The two lower families of curves correspond to procedures starting from configurations obtained using the Belief 
Propagation algorithm of refjsO]: the initial advantage seems eventually lost. 

Before concluding this section, we wish to compare for this case the present algorithm with the 'Belief Propagation' 
(BP) algorithm [s^]. We start with a configuration obtained with BP at a given a (which we shall try to make as 
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large as possible [42|, and then improve the procedure by adding links one by one. The result is shown on the right of 
111! at the beginning there is a very fast progress, but the asymptotic a* is not significantly different - at least within 
our precision. We conclude that starting from a BP solution does not seem to improve the scaling with the system 
size. 

To explain this, we can think in the flood analogy of BP as starting us somewhere in the middle of an island. As the 
level of water keeps on going up, at the beginning we have little to do to stay dry, and the program proceeds rapidly. 
Only when the many-dimensional shore reaches us do we begin to move in a complex manner, and the system slows 
down. 



Scalings with system size 

By performing at each step Walk-COL starting from a site at the end of a new link, we have used on average a 
certain function G{a* — a) color flips each time we add one link, a quantity of order one. The program 'time' t{a) 
(the total number of steps up to connectivity a) thus increases by G{a* — a) each time that a increases by 1/A'^: 

4^ = NG{a* - a) (IV.8) 
da 

and t{a) is N times the integral of G. Thus, the curves t{a)/N for different N should collapse, which we check in 
Fig. [TOl The fact that the total time scales with holds to the extent that the number of rearrangements is, for 
large graphs, independent of the graph size. 

Note that nothing guarantees that G{a* — a), defined on the basis of Walk-COL, is the minimum average number 
of flips needed to make the configuration satisfied after adding a link, but just an upper bound. On the other hand, in 
a locally tree-like structure, the maximal number of moves with any program is to try all the flips of sites at distance 
along the graph smaller or equal than G(a* — a), that is ~ a'*^'" still a quantity independent of N. Thus, the 
fact that the local structure of the graph is tree-like has helped us to obtain a program linear in TV. 

If we did not know that we had a tree-like structure — for example if some loops are present — we would have to 
explore all configurations in phase-space with a given number of flips, trying all the phase space points at progressively 
larger phase-space distances until satisfaction is restored. In this way, G{a* — a) flips would be obtained after at most 
^ ]\[iG{a -a) i^j-ials, and the total integrated time would be: 

Jo \ln N J qG' {a* ~ a) 

This is the absolute worse the program will do for the random graphs discussed here, not using either Walk-COL or 
the knowledge of the local tree- like structure. Below a*, the performance is still a power law in N. 



E. Marginality, hard fields and rattlers. 

As we have seen in the previous sections, when we add links to the Coloring Problem (or clauses to the SAT problem) 
each time the number of changes needed to keep the system satisfied on average increases. This happens because as 
we approach the point a* larger and larger clusters of vertices 'have to be moved together', changes that keep the 
system satisfied have to be done in a correlated manner in each cluster. The same happens with the sphere-packing 
problem: the clusters of particles that are in contact, and have to be displaced together at each step, increases with 
packing and at the critical value of Tq the whole structure percolates. By the time a* (or the J-point, in a packing 
problem) is reached, the correlation diverges, and the algorithm virtually stops. Some important remarks are: 

i) the density of links at which this happens depends on the procedure, the point a'"""'^ obtained at equilibrium need 
not coincide with the one obtained through the sequence of additions of links we have been doing here. (This is why 
in Figure [TT] we have tried to compare the divergence obtained by addition of links starting from low link density, with 
the divergence obtained starting from a very connected graph colored with a BP program. Admittedly, the difference, 
if any, is not large) . For a particle system, the packing fraction at which this percolation happens also depends on the 
procedure. 

a) for the overwhelming majority of runs of the algorithm, when the level a* is reached, the addition of an extensive 
number of links requires a divergent number of re-colorings. However, it will generally happen that a finite fraction 
of vertices can still be recolored in many ways, without destroying the satisfaction. For example, the intermediate 
vertices in a short loop made of a succession of vertices of connectivity two can be recolored in many ways. Similarly, 
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when a particle system reaches Tq, there still are particles, or groups of particles (the 'rattlers') which can move inside 
cages made by the others. 

To summarize: the SAT, Coloring and Packing problems all have critical situations that are process-dependent in 
which a subset of the elements form a 'backbone' subjected to 'hard fields' from their neighbors and 'rattlers' that 
have still freedom. This point corresponds to the EASY/HARD transition for the Recursive Incremental SAT (or Col) 
algorithm described here. If we look at the pseudoenergy of a state, with large probability the number of configurations 
at the critical level of difficulty is still large (due to the freedom of rattlers) , but it disappears completely at a level 
of difficulty just higher. In other words: seen from the point of view of the pseudo-energy, the minima have a 'flat 
bottom' consisting of the rattler configurations [69| . 



V. THE J-POINT 

In a series of papers fll . []3 the highest density configuration of hard spheres reached by 'inflating' as in the section 
IV A have been studied [Sa]. The point where this happens has been christened 'J-point'. As we noted above, they 
did not consider a slow annealing, as in the Lubachevsky-Stillinger procedure, because this would make the system 
evolve in an annealing-time dependent manner, ultimately allowing it to crystallize. For spheres in three dimensions 
the J-point packing fraction turns out to be very close to the one quoted as 'Random Close Packing' 0, and whether 
one can identify J-point and Random Close Packing is a debated question [sO] . The interest of the J-point lies at any 
rate in the fact that it is critical, it has associated with it a diverging length and soft modes [l|, [53| . There is also 
good evidence that for spherical particles it is isostatic, i.e. that the number of contacts is just the number it takes 
to immobilize the system, without redundant static conditions. 

A question that immediately arises is whether the J-point packing corresponds to the 'ideal glass' phase, that is, 
the best packed amorphous state 'without crystallization'. Putting the question this way, we immediately run into 
difficulties because by considering different degrees of crystallinity, we can obtain a continuum of denser packings 
[4^. The random models studied here allow us to avoid - at least to fix ideas ~ the conceptual complication of 
crystallization, and to concentrate on the glassy aspects [45|. We shall hence first discuss the SAT and Coloring, 
which by construction have no ordered states. On the latter the procedure of Ref. [l|] can be repeated in the 'angle 
packing' version without modifications. 

In SAT and Coloring we have already pointed out that the recursive Incremental procedure of adding difficulty in 
a constraint satisfaction problem yields a critical point with a diverging length, given by the range of rearrangements 
necessary each time to satisfy the constraints (see Ref. [i^, |3l). This point, which we identify as the mean-field 
version of the J-point, also corresponds to the appearance of frozen variables in the state [29| . 

In the framework of the pseudo-energy landscape the procedure leading to the J-point amounts to a deep (zero 
temperature) quench in this complex landscape, starting from a random configuration. In this language, it is an 
infinite temperature inherent structure. Clearly, there is no reason for this procedure to converge to the (glassy) 
ground state (otherwise it would be the ultimate solution to the coloring and SAT problem) : minima with the largest 
basin of attraction are generically not the deepest. 

Running the program several times starting from different initial conditions leads to different end densities [l|, 
distributed around a typical value, the closer the larger the system: The pseudo-energy view allows us to recognize 
this as the standard situation with the inherent structure energy distribution [l^ . 

Another thing to notice is that configuration reached in the J-point is not typical of the given packing fraction, 
just as an inherent structure reached by a quench in an energy landscape is not the typical one of the corresponding 
energy. One way to emphasize this is to envisage the analog of the protocol of Ref. [39|: starting from equilibrium 
configurations at different initial packing fractions (or values of Tq), one applies the inflating procedure of O'Hern 
et al. If the landscape hypothesis advocated here holds, the final packing fraction (or Tq) reached should depend on 
the initial one in much the same way as in Ref. (39j the inherent structure energies depend upon initial equilibrium 
energies (in finite dimensions crystallization effects have to be taken care of [43). It would be interesting to study 
other amorphous configurations with the same packing fraction as the J-point, but reached after different annealing 
protocols, and check whether they are not critical. This is what happens in the mean-field models discussed above: 
their typical configurations at a packing fraction equal to that of the J-point are not critical, even in this situation in 
which there is no possibility of crystalline order. 
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VI. ANALYTICAL COMPUTATIONS OF PERFORMANCE. DYNAMICS AND CAUSAL REPLICA 

CHAINS. 



The statistical mechanical description of complexity, in particular based on the replica trick, concerns the average 
geometric structures in a given subset of phase-space. This information is local, in the sense that it involves the 
structure of typical states and not their whole basin of attraction. Sometimes, this local information is enough to 
infer where a dynamic process starting from a random configuration will go. This is the case for example of dynamics 
of certain mean-field models, where 'marginal' states are chosen. However, it is not clear in general up to what point 
local information of states is sufficient. One alternative that has been widely discussed in the literature, is to solve the 
averaged dynamics (for a review of this approach, see: [isj). Here we shall discuss a different but related approach, 
which is a generalization of several works in the replica literature of the 80's and 90's [5^, [b^, [o^l and in optimization 
problems [38|. 

Let us first consider an energy landscape of discrete variables. We start by choosing a random configuration Si (here 
we use boldface to denote iV-dimensional vectors). Next, keeping this one fixed, we search for another configuration 
S2 at distance Ai from the first. Keeping these two fixed, we choose a third configuration S3 at distance A2 from the 
second, and so on. In this way, we construct a 'causal chain', since the subsequent links do not affect the preceeding 
ones. We have to demand, for example, that each new replica be at an energy as small as possible (subject to its 
constraints): this is done by thermalizing each link at a very low temperature. Next, we have to make the distance 
between links tend to zero, and the number of links to infinity. From the point of view of replica theory, this is jus t 
a generalization of the 'effective potential' method [s^, and is also closely related to the discussion in Rcfs. [13, [el]. 
We have to write a partition fmiction: 

Z = lim ■■. lim / ds^^ds"^^ '-.ds^ (5 (chain) 

{ ni 712 riR 1 

Qi = l 02 = 1 ait = l ) 



Where s^^,S2^ "• are all N dimensional vectors of spins, s^^ is replicated ni times, 712 times, etc. Spins are 
organized in a hierarchical manner: one of the ni replicas s^'^ has an offspring of n2 of the 82^. One the n2 of these 
has in turn an offspring of of the Sg^, and so on. The term (5(chain) is a product of deltas imposing that the r-th 
'offspring' is at a fixed distance from its parent. 

For a constraint satisfaction problem, the problem can be recast as follows. Taking the example of the coloring 
problem, we suppose we have a graph with connectivity per node amax- The links are given by an A'^ x A'^ symmetric 
matrix Jy with elements one and zero. An ordering is introduced by defining the symmetric matrix Cij, with elements 
taken at random with distribution uniform in (0, amax)- The number of miscolorings at connectivity a is then: 

A/(s, a) = ^ e(a - aj)5,^s, (VI.ll) 

ij 

where &{x) is one if x is positive, and zero otherwise. One repeats now the argument as before, for the case of a chain 
of replicas with progressively larger values of a, keeping the total number of errors equal to zero. 

In practice, one can make the computation for chains of a small numbers of links, and extrapolate the result to 
the limit of continuous chain. For systems on the Bethe lattice, the replica calculation should be translated into the 
cavity language, which is more flexible. 



VII. CONCLUSIONS 



We have adopted a unified point of view of hard-particle glasses and the constraint-satisfaction problems of computer 
science and taken both to a (pseudo) energy landscape setting. On the one hand, the glass literature has long been 
a source of inspiration and methods to attack satisfaction problems. On the other hand, the statistical mechanics of 
optimization problems provides nontrivial yet solvable models for which questions on the jamming transition can be 
answered exactly. 

In both cases, there are questions — such as a rigidity (hard field) threshold, the presence of 'rattler' and a backbone 
of fixed particles (or Boolean variables, or colors) — peculiar to the fact that the task is posed as a zero temperature 
problem of optimizing parameters while respecting a constraint. The pseudo-energy language helps to bring back 
many of these results to a problem of relaxation in a complex landscape. 
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Satisfaction 

The statistical mechanic point of view of satisfaction has been mainly concentrated on computation of static 
'snapshot' properties of phase-space. This approach has been very fruitful and has suggested powerful algorithms 
[J. Here, instead, we take the approach of defining an algorithm whose properties can in principle be computed 
analytically, at least in the same cases for which a static solution is possible. This change of perspective reflects 
a similar passage from a static to a dynamic approach in the glass literature p^ . and has also been pursued in 
the statistical mechanics of optimization (58| . In this paper we have only stated but not completed the analytical 
calculation, but we have shown numerically that problems are easy beyond the 'clustering transition', and with the 
Coloring Problem that in some cases even beyond the 'Kauzmann' one. Indeed, one of the conclusions is that a rather 
excessive importance has been attributed to the clustering (dynamic) transition point: this is the point up to which 
any program can easily sample all configurations (ssj . but not the last point at which it can easily obtain at least 
one configuration — this happens at a*. It is then, upon reflection, not so surprising that relatively straightforward 
algorithms such as Walk-SAT [4l[ find easily solutions almost up to the satisfiability threshold. 

Jamming transition 

When the J-point was introduced [l[ and shown to correspond to a packing fraction close to the empirical value 
of 'random close packing', the question arose of its meaning in terms of the glass state. What the present approach 
underlines (but was already implicit in llOll ') , is that the nature of this point is the same as that of a zero-temperature 
quench in a rugged energy landscape [6ll |. Deeper, more compact, points can be reached with other annealing 
procedures - even in models without a crystalline state , as the examples we have considered show. This suggests 
a picture of the transition in the temperature/shear-rate/dcnsity space [5ll |. rather than as a single surface, as a 
multi-layered onion, each level being reached after a different protocol. 

There is density at which a hard particle system has an experimentally unaccessible relaxation timescale: this 
happens at a density distinctly smaller than the J-point one. Indeed, the present analysis shows that the J-Point is 
necessarily denser than the Mode-Coupling transition density, in models in which such a transition exists and is sharp. 
On the other hand, the question of which point is denser, the J-Point or the Kauzmann transition point depends on 
the system and the dimensionality - again based on what can be inferred from models for which the latter transition 
can be shown to exist (g^]. Note that, if the J-point is denser than the glass transition point, this is at the expense 
of being out of equilibrium. 

An interesting aspect of the J-point is that it is critical. Here we have found that the same is also true for mean- 
field models (as had already been found in the (/-core problem [6^), but this criticality does not correspond to the 
thermodynamic glass transition itself (which in mean-field exists), but is that of the zero-temperature 'threshold' 
states [26| already familiar from the mean-field glassy dynamics. In fact, in any of the problems discussed here, if the 
procedure of approach to the J-point is stopped when the correlation length has reached a certain large value, and a 
thermalization at fixed Tq (or a) is applied subsequently, as the system tends towards equilibrium the pressure drops 
and the correlation length diminishes, rather than growing. 

The study of problems with disorder raises doubts on the fact that isostaticity of a frictionless particle system is 
a necessary (apart from sufficient [ij, [s^l, although see [6^) condition for criticality. This may be checked just 
by testing the J-point procedure with a packing of non-spherical (e.g. ellipsoidal) particles, which are generically 
hypostatic [13]: if the J-point for ellipsoids turns out to be critical, then it will be clear that isostaticity is not a 
necessary condition for criticality. 

Finally, the analogy between particle and satisfaction problems suggests that one may perhaps build a mean-field, 
'Random First Order' theory of rigidity (and to a certain extent plasticity) by considering systems which develop 
'hard fields', and subjecting them to driving, nonconservative forces. The Coloring Hamiltonian, or its 'angle packing' 
version, is a good candidate for such a strategy. 
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